Broadcasting method and broadcast signal receiving apparatus

ABSTRACT

A broadcasting method and a broadcast signal receiving apparatus is provided wherein auxiliary information, such as video and audio data for supplementing broadcast information presented through a broadcasting network in an interlocked manner can be watched and listened to by the viewer. Broadcast information received from a broadcasting network is temporarily stored in a storage means of a broadcast signal receiving apparatus and a recording medium for storing auxiliary information used as a supplement to the broadcast information is mounted on the broadcast signal receiving apparatus. The broadcast information and the auxiliary information are then played back in an interlocked manner with a predetermined timing. As a result, a function to present information which cannot be accommodated in a broadcast frame and a parental function can be implemented.

The present application is a continuation of application Ser. No.11/826,560, filed Jul. 17, 2007; which is a continuation of applicationSer. No. 09/380,784, filed Sep. 9, 1999, now U.S. Pat. No. 7,272,843,the contents of which are incorporated herein by reference.

The present invention relates to a broadcasting method and a broadcastsignal receiving apparatus for presenting video information, such as apicture, through a broadcasting network; and more particularly, theinvention relates to a broadcasting method and a broadcast signalreceiving apparatus wherein information presented through a broadcastingnetwork and auxiliary information presented through other media, such aspackage media, can be watched and listened to in an interlocked manner.

BACKGROUND OF THE INVENTION

In a conventional system, video/audio information is edited so as to bebroadcasted during a limited broadcasting period of time set as aprogram. Thus, the viewer can watch and listen to only that part of suchvideo/audio information that is actually broadcasted by a broadcastingstation. For example, in the case of a news program, it may be that onlypart of gathered news information can be watched and listened to due tosuch editing. In the case of a movie program, on the other hand, onlypart of a full length movie as presented at a movie theater can bewatched during the period of a movie program, which is usually shorterthan the show time of the movie at the movie theater.

Meanwhile, in recent years, there has been implemented a databroadcasting system whereby text data related to video/audio informationis broadcasted as data in addition to the video/audio information byappending the text data in the blanking signal period of the videoinformation. For example, systems called IT-vision and Intel-Intercastare known. In these data broadcasting systems, text data is described ina predetermined language, such as an HTML (Hyper Text Markup Language),in a video signal period of time, which is not used in the videotransmission to be broadcasted, along with the video signal at the sametime. An example of the signal period of time used for this purpose isthe VBI (vertical blanking interval). In the broadcast signal receivingapparatus, incoming video/audio information and text data are separatedfrom each other. The video/audio information is played back as it is,while the text data is displayed on a screen of the display unit as amessage related to the video information, such as a text, a staticpicture or a graphic. In this way, the viewer is capable of not onlywatching and listening to the video/audio information, but also iscapable of obtaining information on its program at the same time.

In a data broadcasting system whereby auxiliary information isbroadcasted by using the VBI with a low information transmissioncapacity, however, there is a limit on the amount of auxiliaryinformation that can be transmitted at one time, making it impossible tobroadcast a large amount of data, such as video information, asauxiliary information. While video information and related informationare broadcasted at the same time, they are independent of each other;and thus, it is difficult to utilize the related information in aprocess of synthesizing video information, such as a search for originalinformation based on the related information and correlation of a scenecut out from a movie with the original movie.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide the viewer withvideo/audio data which can not be included in broadcast information inthe form of auxiliary information by adopting a broadcasting proceduredifferent from the normal broadcasting procedure, wherein the auxiliaryinformation is synthesized automatically in the broadcast signalreceiving apparatus with the broadcast information to generate finalinformation which appears to the viewer as if the final information hadbeen broadcasted as a continuous single program.

According to the present invention, broadcast information received bythe broadcast signal receiving apparatus during a predetermined programperiod of time is played back in a manner interlocked with auxiliaryinformation received in advance and held in the receiving apparatus.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be described with reference tothe accompanying drawings, wherein:

FIG. 1 is a schematic block diagram of a package media interlockedbroadcasting/watching/listening system;

FIG. 2 is a block diagram of a 1st embodiment implementing a broadcastsignal receiving apparatus;

FIG. 3 is a diagram which shows the data format of broadcast informationreceived by the 1st embodiment;

FIG. 4 is a diagram which shows the data format of auxiliary informationreceived by the 1st embodiment;

FIG. 5 shows a typical data format of an auxiliary information controltable;

FIG. 6 shows a typical video recording control table; FIGS. 7(A) to 7(D)are diagrams which show a sequence of user operations;

FIG. 8 is a diagram which shows an example of broadcast information andFIGS. 8(A) and 8(B) are diagrams which illustrate different algorithmsadopted by a video synthesizing unit for synthesizing broadcastinformation and auxiliary information;

FIG. 9 is a block diagram of a 2nd embodiment implementing a broadcastsignal receiving apparatus;

FIG. 10 is a diagram which shows the data format of broadcastinformation and auxiliary information received by the 2nd embodiment;and

FIG. 11 is a diagram which shows the data format of auxiliaryinformation handled in a 3rd embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 will be referred to in order to explain the concept of a packagemedia interlocked broadcasting/watching/listening system. In the system,the contents of a broadcast program are created by a contents provider900 and the contents are partly delivered to a broadcasting station 901as broadcast information 201. The broadcasting station 901 broadcaststhe broadcast information 201 through an infrastructure, such as asatellite, a ground wave facility or a CATV. The contents provider 900also distributes other information related to the broadcast information201, through the Internet and a package media, as auxiliary information301. The viewer side (at a home 902) receives the broadcast information201 and the auxiliary information 301 through a broadcast signalreceiving apparatus 100 and displays the information on a monitor. Inthe package media interlocked broadcasting/watching/listening system,auxiliary information 301 supplements broadcast information 201. Bothauxiliary information 301 and broadcast information 201 include indexes305 and 203, respectively, for associating the former with the latterand for specifying how and with what timing the former supplements thelatter.

The following is a description of an example to which the package mediainterlocked broadcasting/watching/listening system is applied. Whendistributing the contents of a large size theater movie, scenes in themovie are cut out by the broadcast information contents provider 900 tocorrespond to a length adjusted to a predetermined broadcast time. Thecontents provider 900 delivers the movie-through-the-cutting-process toa broadcasting station, as broadcast information, which is to bebroadcasted to various homes. On the other hand, the cut out scenes aresent to the homes as auxiliary information through other means, such asthe Internet, package media and data broadcasting systems. A broadcastsignal receiving apparatus at each home receives the broadcastinformation and the auxiliary information and, thereafter, displays bothpieces of information on a monitor in an interlocked manner based onindexes appended to both. This enables the viewer to watch and listen tothe contents of the complete movie. In a news program, part of the newsnot broadcasted can also be presented to the viewer as auxiliaryinformation. As a result, there is born a new information distributionbusiness for distributing and delivering more detailed subjects, whichare not broadcasted as a news program, as auxiliary information in theway described above.

The package media interlocked broadcasting/watching/listening system canalso be applied to parental control. A parental control functionprevents pictures and expressions of the original work which may beinappropriate for juvenile viewers from being displayed. To be morespecific, portions of the original work for general viewers arebroadcasted as broadcast information whereas scenes for adult viewersare only delivered separately as auxiliary information, typically byusing package media. This allows the viewer to select whether or notsuch auxiliary information is to be displayed at the time of viewing abroadcast program, implementing the parental control function toappropriately exclude such scenes from displayed information. Thus, thechances that scenes inappropriate for juvenile viewers are inadvertentlywatched and listened to are eliminated.

Moreover, the package media interlocked broadcasting/watching/listeningsystem can also be applied to broadcast advertisements (CMs). Moreparticularly, by delivering a CM such as a product advertisement asauxiliary information, the CM can be played back properly insynchronization with a program. As a result, the number of opportunitiesin which a product advertisement is presented to the viewer in additionto the CMs included in the broadcast information can be increased. Bysetting an auxiliary information CM time in addition to the traditionalCM time, the broadcasting station is able to increase its advertisementrevenue. In addition, the CM provider is capable of appending a CMrelevant to a program to the program broadcasted by a broadcastingstation that does not broadcast the CM. For example, an index of thename of a program included in broadcast information can be used to playback an advertisement of a product, such as a car appearing in theprogram, from a CD-ROM with timing specified by the index. In this way,the CM provider is capable of letting the viewer watch and listen to aCM even if the broadcast station does not broadcast the CM.

The following is a description of an application of computer graphics tothe package media interlocked broadcasting/watching/listening system. Ifthe playback apparatus includes a processor that is capable of executinga program, auxiliary information played back in a manner interlockedwith an operation to display broadcast information will not be limitedto the video and audio data described above. For example, a CG program,i.e., a program displaying computer graphics (CG), can be provided asauxiliary information. That is, a CG program can be delivered asauxiliary information by using package media. Invocation of a CG programis triggered by index information included in the broadcast information.By providing a parameter required for a CG program from the broadcastinformation, a CG character moving synchronously with the broadcastingis allowed to appear on a displayed screen of the broadcast information.Such an application is particularly effective for programs, such asanimation. The viewer is capable of carrying out operations, such asreplacing some characters appearing in an animation program with CGcharacters selected from package media. If the viewer is also capable ofcreating a CG character, the viewer will be able to enjoy seeing a CGcharacter created by himself or herself appearing and acting in abroadcasted program. Such a joy can not be realized by the conventionalTV broadcasting.

In the mean time, standardization, including one known as MPEG4, hasmade progress in recent years. In the video and audio encoding field, atechnology called content based encoding has been implemented. Thecontent based encoding is a technology wherein video materials (orobjects) included in video data is encoded independently, and, on thereceiving side, the objects are decoded individually and finallysynthesized before being displayed. In this technology, since onlyinformation on objects that change in the video data is transmitted, notonly is it possible to considerably reduce the amount of information tobe transmitted, but other advantages can also be attained by a videodata producer. The other advantages include a capability of creatingentirely new video data by a combination of pieces of video data ofobjects photographed individually. With this content based encoding usedin the transmission of broadcast information, if the package mediainterlocked broadcasting/watching/listening system using a broadcastsignal receiving apparatus provided by the present invention is adopted,video data of some objects included in a broadcast signal can bereplaced by video data of other objects recorded in a piece of exactlycorresponding information recorded on package media.

For example, video data of a 1^(st) person appearing in a program andvideo data of a 2^(nd) person other than the 1^(st) person are eachsubjected to content base encoding with the 2^(nd) person closelyimitating behaviors of the 1^(st) person. The encoded data of the 1^(st)person is broadcasted as part of broadcast information, while theencoded video data of the 2^(nd) person is recorded on package media fordistribution. If only the broadcast information is received and playedback, the 1^(st) person will appear in a program as it is along withother objects, such as a background. If the broadcast information isplayed back in a manner interlocked with the package media, on the otherhand, the viewer will be capable of watching and listening to video datawherein the 1^(st) person is replaced by the 2^(nd) person.

The following is a description of a broadcast signal receiving apparatusof the present invention which makes the package media interlockedbroadcasting/watching/listening system described above feasible.

FIG. 2 shows the system configuration of a 1st embodiment implementing abroadcast signal receiving apparatus 100 provided by the presentinvention. As shown in the figure, the broadcast signal receivingapparatus 100 comprises:

a broadcast signal receiving means 101 for receiving broadcastinformation 201;

a storage means 106, such as a hard disc drive, for temporarily storingbroadcast information 201 received by the broadcast signal receivingmeans;

a removable media drive 104 for inputting auxiliary information 301;

an input means 107 for inputting a user intention;

a processor 102 for carrying out processing to convert broadcastinformation 201 and auxiliary information 301 into a video signal;

a display unit 108, such as a color monitor, for displaying information;

a display means 103 for converting video data generated by the processor102 into a signal to be displayed on the display unit 108; and

a memory unit 105 used for storing programs to be executed by theprocessor 102 and for temporarily storing data in the course of signalprocessing carried out by the processor 102.

While it is desirable to put a mountable as well as removablerandom-access medium, such as a CD-ROM, a DVD and an MO, in theremovable media drive 104, a sequential medium, such as a tape is alsoacceptable as well.

Broadcast information 201 is stored in the storage means 106 in programunits. When the viewer watches and listens to a program, broadcastinformation 201 stored in the storage means 106 is transferred to theprocessor 102, which then carries out processing, such as decoding, onthe broadcast information 201 using the memory unit 105 as a work area.A result of the processing is then output to the display means 103. Ifan index is extracted from the broadcast information 201 during theprocessing, auxiliary information 301 indicated by the index is read outfrom a medium mounted on the removable media drive 104. Processing isthen carried out on the auxiliary information 301 and a result of theprocessing is output also to the display means 103.

It should be noted that the means used for furnishing auxiliaryinformation 301 are not limited to media such as a CD-ROM. Auxiliaryinformation 301 can also be supplied to the broadcast signal receivingapparatus 100 byway of a network such as the Internet or a databroadcasting system such as a satellite broadcasting system or a groundwave broadcasting system. In this case, auxiliary information 301received from a partner on the Internet or a broadcasting station of thedata broadcasting system is transferred to the storage means 106 or thememory unit 105 shown in FIG. 2 to be stored therein. In either case,the auxiliary information 301 stored in the recording means 106 or thememory unit 105 is played back in a manner interlocked with broadcastinformation 201.

FIG. 3 shows the data format of broadcast information 201 received bythe 1st embodiment. As shown in the figure, a plurality of programindexes 203 are inserted between pieces of video and audio data in eachprogram. Each of the program indexes 203 includes information, such as aprogram ID 204 for uniquely identifying a broadcast program, a programname 205 given to the program, a start time 206 of the program, an endtime 207 of the program and one or more auxiliary information IDs 208each for identifying a piece of auxiliary information 301 to beinterlocked during an operation to play back the program. In the case ofan analog broadcast signal, a program index 203 is embedded in a VBIregion of the video signal by using the ordinary NTSC system videosignal. In the case of a digital broadcast signal, on the other hand, aprogram index 203 is broadcasted as a control packet other than thevideo signal.

It is desirable to insert a plurality of program indexes 203 into aprogram. This is because, if a program index 203 is inserted into onlythe head of a program, no program index 203 can be acquired in arecording or playback operation of the program starting from a positionafter the single program index 203, making it impossible to play backthe program in a mode interlocked with auxiliary information 301 withthe auxiliary information 301 used as a supplement to broadcastinformation 201. By inserting a plurality of program indexes 203 into aprogram, however, this problem can be solved. A program index 203 mayinclude auxiliary information IDs 208 of all pieces of auxiliaryinformation 301 to be interlocked with the program or include auxiliaryinformation IDs 208 of only pieces of auxiliary information 301 to beinterlocked with the program starting from a point into which theprogram index 203 is inserted.

FIG. 4 shows the data format of auxiliary information 301 received bythe 1st embodiment. As shown in the figure, auxiliary information 301comprises a plurality of individual auxiliary information pieces 304 andan auxiliary information control table 303 for centrally controlling theindividual auxiliary information pieces 304. Each of the individualauxiliary information pieces 304 comprises audio/video data 306 and anauxiliary information index 305 including information for identifying apartner program to be supplemented by the video/audio data 306. Theauxiliary information index 305 comprises an auxiliary information ID307 for distinguishing the individual auxiliary information pieces 304from each other, the title 308 of the individual auxiliary informationpiece 304, a partner program ID 309 for identifying a partner program tobe supplemented by the video/audio data 306, a supplementation starttime 310 with the beginning of a partner program to be supplementedtaken as a base, supplementation end time 311 and time lengthinformation 312 of the individual auxiliary information piece 304.

FIG. 5 shows a typical data format of the auxiliary information controltable 303. As shown in the figure, the auxiliary information controltable 303 is a single collection of attributes of auxiliary informationindexes 305 of all individual auxiliary information pieces 304 andstorage locations 401. That is, each line of the auxiliary informationcontrol table 303 comprises attributes of an auxiliary information index305 and one storage location 401. A storage location 401 is an addressin a medium or a storage device at which the video/audio data 306 of theindividual auxiliary information piece 304 is stored. The auxiliaryinformation indexes 305 cataloged in the auxiliary information controltable 303 are sorted by partner ID 309. Note that it is desirable toautomatically copy the auxiliary information table 303 into the storagemeans 306 at the time the medium storing the auxiliary information table303 is mounted on the removable media drive 104. With the auxiliaryinformation control table 303 copied into the storage means 106, asearch of the table 303 for a desired piece of information can becarried out at a high speed.

Next, a package media interlocked broadcasting/watching/listening methodwill be explained. The package media interlocked broadcasting/iswatching/listening method comprises the steps of:

carrying out a recording operation to record broadcast information 201into the storage means 106; and

carrying out a synthesis/playback operation to play back a program whilesupplementing the broadcast information 201 with individual auxiliaryinformation pieces 304.

First of all, the recording operation will be explained with referenceto FIGS. 2 to 6. The viewer records broadcast information 201 into thestorage means 106 in the same way as a TV program is recorded into a VTR(video tape recorder). In the case of a storage means 106 implemented bya digital information storage means, such as a hard disc, an analogbroadcast signal is converted into a digital signal, such as an MPEGstream, by a video capture in the broadcast signal receiving means 101,and the resulting digital signal is then stored in the storage means106. The viewer is allowed to specify a recording time in the case ofreserved recording or unattended recording, or to carry out a manualoperation to start and end the recording. The processor 102 receives andstores incoming broadcast information 201 in accordance with a requestcoming from a timer not shown in the figure in the case of unattendedrecording or a request made by the viewer in the case of manualrecording. At that time, the broadcast signal receiving apparatus 100catalogs the recorded program into a video recording control table 501,a table for cataloging recorded programs, like one shown in FIG. 6. Asshown in the figure, each line in the video recording table 201comprises a program ID 204 as well as a program name 205 extracted froma program index 203 of the broadcast information 201, a recording starttime 504, a recording end time 505 and a recording location 506 or anaddress in the storage means 106 indicating the beginning of an area forstoring the broadcast information 201. It should be noted that therecording start time 504 and the recording end time 505 shown in thevideo recording control table 501 are times with a program start time206 of the recorded program taken as a reference. That is to say, therecording start time 504 and the recording end time 505 are obtained bysubtracting the program start time 206 from the time the recording isactually started and the time the recording is actually ended,respectively.

Next, a typical synthesis/playback operation will be explained withreference to FIGS. 2 to 7. In order to obtain individual auxiliaryinformation pieces 304 to be used for supplementing a program, arecording medium containing auxiliary information 301 is mounted on theremovable media drive 104. When the recording operation described aboveis not carried out, the broadcast signal receiving apparatus 100 carriesout a through operation wherein the processor 102 processes incomingbroadcast information 201 and displays the processed broadcastinformation on the display unit 108 by way of the display means 103 asis the case with an operation to receive an ordinary televisionbroadcast program. For control purposes, the processor 102 displays aplayback button 602 on the screen of the display unit 108 as shown inFIG. 7A.

When the viewer selects the playback button 602 by using a cursor 601 orpresses a playback request button provided on the input means 107, suchas a remote controller, the broadcast signal receiving apparatus 100displays a recorded program menu 603 created at that time from the videorecording control table 501 on the display unit 108 in order to let theviewer select a program recorded in the storage unit 106 to be playedback. The viewer then selects a desired recorded program from therecorded program menu 603 as shown in FIG. 7B by specifying the programname thereof. When the name of the desired recorded program isspecified, the video recording control table 501 is searched for thename 205 of the desired program and the program ID 204 thereof isacquired.

Next, the processor 102 forms a decision as to whether or not auxiliaryinformation 201 can be acquired from a recording medium mounted on theremovable media driver 104. The decision is formed by determiningwhether or not the program ID 204 of the selected program acquired fromthe video recording control table 501 exits as a partner program ID 309in the auxiliary information control table 303 which was copied from therecording medium into the storage means 106 when the recording mediumwas mounted on the removable media drive 104 for the first time. If theoutcome of the decision indicates that auxiliary information 201 can notbe acquired, that is, if no partner program ID 309 in the auxiliaryinformation control table 303 matches the program ID 204,supplementation of the program by auxiliary information 201 is ignored.The processor 102 then searches the video recording control table 501using the program ID 204 as a key for the recording location 506.Subsequently, the processor 102 reads out the program from the location506 to play it back. If the outcome of the decision indicates thatauxiliary information 201 can be acquired, that is, if a partner programID 309 in the auxiliary information control table 303 matches theprogram ID 204, on the other hand, the matching partner program ID 309is used for fetching attributes 307 to 312 and storage locations 401 ofeach auxiliary information index 305 associated with the desired programto be played back from the auxiliary information control table 303. Anauxiliary information menu 604, shown in FIG. 7C, is then displayed onthe screen. It should be noted more than one auxiliary information index305 in the auxiliary information control table 303 may have a partnerprogram ID 309 matching the program ID 204, that is, more than oneindividual auxiliary information piece 304 may be associated with thedesired program. As shown in FIG. 7C, the auxiliary information menu 604includes a table of individual auxiliary information pieces 304associated with the desired program and a ‘Select all’ button which isoperated if it is desired to specify all of the individual auxiliaryinformation pieces 304. Beneath the auxiliary information menu 604, thefollowing buttons 605 to 607 are also displayed on the screen shown inFIG. 7C. The ‘Play back auxiliary information’ button 605 is selected ifit is desired to play back only the individual auxiliary informationpieces 304. On the other hand, the ‘Playback program’ button 606 isselected if it is desired to play back the broadcast information 201only, ignoring the individual auxiliary information pieces 304. Finally,the ‘Synthesis/Playback’ button 607 is selected if it is desired to playback the program by supplementing the broadcast information 201 of theprogram with the individual auxiliary information pieces 304.

Assume that the viewer selects an ‘auxiliary information 1’ itemrepresenting one of the individual auxiliary information pieces 304listed on the auxiliary information menu 604 and the‘Synthesis/Playback’ button 607. In this case, the processor 102searches the video recording control table 501 using the program ID 204as a key for a recording location 506, a recording start time 504 and arecording end time 505 of the specific broadcast information 201 to beplayed back. Then, the processor 102 searches the auxiliary informationcontrol table 303 for a recording location 401 of the individualauxiliary information piece 304 corresponding to the ‘auxiliaryinformation 1’ selected from the auxiliary information menu 604 to besynthesized and played back with the specific broadcast information 201,a supplementation start time 310 and a supplementation end time 311 ofthe individual auxiliary information piece 304 used in the synthesizedplayback.

The recording start time 504 of the specific broadcast information 201to be played back is compared with the supplementation start time 310 ofthe individual auxiliary information piece 304 to be synthesized. Sincethe recording start time 504 and the synthesis start time 310 are bothgiven with the program start time taken as a reference, the comparisoncan be decided to be justifiable. Video and audio data of theinformation with the smaller start time is played back as shown in FIG.7D.

An individual auxiliary information piece 304 is played back as follows.The processor 102 compares the supplementation start time 310 of theindividual auxiliary information piece 304 with the contents of a timeralso taking the program start time as a reference all of the time.

When the contents of the timer become equal to the supplementation starttime 310, the operation to play back the broadcast information 201 istemporarily suspended and an operation to play back the individualauxiliary information piece 304 is started. When the operation to playback the individual auxiliary information piece 304 is finished, theoperation to play back the broadcast information 201 is resumedbeginning with a portion at the supplementation start time 310 or thesupplementation end time 311 of the individual auxiliary informationpiece 304. If the operation to play back the broadcast information 201is resumed beginning with a portion at the supplementation start time310, the individual auxiliary information piece 304 is merely insertedinto the broadcast information 201. A method of synthesis to insert anindividual auxiliary information piece 304 into broadcast information201 is referred to as an insertion type synthesis method. If theoperation to play back the broadcast information 201 is resumedbeginning with a portion at the supplementation end time 311, on theother hand, the individual auxiliary information piece 304 replaces partof the broadcast information 201 between the supplementation start time310 and the supplementation end time 311. A method of synthesis toreplace part of broadcast information 201 with an individual auxiliaryinformation piece 304, is referred to as a replacement type synthesismethod. The insertion type synthesis method and the replacement typesynthesis method will be exemplified in concrete terms later.

If the ‘Play back auxiliary information’ button 605 shown in FIG. 7C isselected by the user, only individual auxiliary information pieces 304are played back sequentially one piece after another beginning with theone with the smallest supplementation start time 310 shown in theauxiliary information control table 303. If the ‘Play back program’button 606 shown in FIG. 7C is selected by the user, on the other hand,only the broadcast information 201 selected from the menu shown in FIG.7B is played back. In addition, interface buttons 608 resembling buttonson a remote controller used for controlling an ordinary VTR aredisplayed on the screen as shown in FIG. 7D. The interface buttons 608allow the user to specify operations to start, halt, temporarily halt,fast forward and rewind the playback of the video data. When the userspecifies the operation to halt the operation by operating one of theinterface buttons 608, the broadcast signal receiving apparatus 100terminates the operation to play back video data and resets the timer.

In the above description, it is assumed that a method is used wherebybroadcast information 201 is recorded into the storage means 106 shownin FIG. 2 in program units, and, in an operation to play back a program,broadcast information 201 of the program is synthesized with auxiliaryinformation 301. If the read/write speed of a hard disc drive used asthe storage means 106 is not high enough to keep up with the recordingand playback operations, however, part of the memory unit 105 can beused as a buffer between the processor 102 and the storage means 106.The buffer allows the operations to record broadcast and play backinformation 201 into and from the storage means 106 to appear as if theywere carried out concurrently. As a result, the package mediainterlocked broadcasting/watching/listening method can be applied to aprogram which is currently being broadcasted.

The buffer cited above works as follows. Broadcast information 201received by the broadcast signal receiving means 101 is stored in a 1stbuffer area in the memory unit 105. As the amount of stored broadcastinformation 201 reaches a predetermined value, the broadcast information201 is transferred to the storage means 106 at one time as a batch.While the transfer is being carried out, the operation to storebroadcast information 201 received by the broadcast signal receivingmeans 101 can be continued. In the mean time, broadcast information 201transferred to and recorded in the storage means 106 is transferred backto a 2nd buffer area in the memory unit 105 in appropriate units to beplayed back. Pieces of broadcast information 201 in the 2nd buffer areaare sequentially processed one piece after another by the processor 102and output to the display means 103. In this way, while currentlybroadcasted information 201 is being recorded in the storage means 106,a playback operation can be continued with a time delay to a certaindegree. While such simultaneous recording and playback operations arebeing carried out in this way, auxiliary information 301 can be used asa supplement in accordance with program indexes included in thebroadcast information 201 being played back. Time delays result due tosupplementation by using auxiliary information 301. However, such timedelays can be absorbed by the simultaneous recording and playbackoperations. If the insertion type synthesis method described above istaken as a technique of supplementation, the time delay betweenbroadcast information being recorded and the information 201 beingplayed back is widened. If the capacity of the storage means 106 islarger than a size corresponding to the time delay, however, there willbe no missed information at all.

FIG. 8 shows an example of broadcast information, and FIGS. 8A and 8Bshow two different algorithms adopted in a synthesis of broadcastinformation 201 and auxiliary information 301. To be more specific, FIG.8A is a diagram showing the insertion type synthesis method, and FIG. 8Bshows the replacement type synthesis method. As shown in FIG. 8A, in thecase of the insertion type synthesis method, the broadcast signalreceiving unit 100 plays back auxiliary information 1 denoted byreference numeral 306 after playing back video and audio data 202 ofbroadcast information 201 denoted in FIG. 8 by notation M1 up to a pointof time T1. After the operation to play back auxiliary information 1 isfinished, the operation to play back the video and audio data 202 of thebroadcast information 201 is resumed starting with a portion M2 at thepoint of time T1. In this way, auxiliary information 1 is merelyinserted at the point of time T1 into the video and audio data 202 ofthe original broadcast information 201. In the case of the replacementtype synthesis method shown in FIG. 8B, on the other hand, as theoperation to playback the auxiliary information 1 is finished at a pointof time T2; the operation to play back the video and audio data 202 ofthe broadcast information 201 is resumed starting with a portion M3 atthe point of time T2. In this way, auxiliary information 1 replaces theportion M2 between the points of time T1 and T2 of the video and audiodata 202 of the original broadcast information 201.

It should be noted that, in an effort made in the actual implementationto adjust timing in a playback operation, auxiliary information 301 isread out earlier than an accurate start time and stored in a cachememory. By doing so, the auxiliary information 301 can be played backwith good timing without causing the viewer to feel a sense ofincompatibility. In addition, if interframe encoding is carried out toproduce an MPEG stream, a frame on an encoding boundary, that is, an Iframe in the case of MPEG encoding, is adjusted to coincide with aboundary between broadcast information 201 and auxiliary information301.

Next, a 2nd embodiment implementing a broadcast signal receivingapparatus provided by the present invention will be explained. In thecase of the 2nd embodiment, auxiliary information 301 is alsotransmitted in the same way as broadcast information 201 instead ofbeing provided by means of a recording medium. FIG. 9 shows the systemconfiguration of the 2nd embodiment. Since auxiliary information 301 isalso transmitted as is the case with broadcast information 201, as shownin the figure, the 2nd embodiment is obtained by omitting the removablemedia drive 104 from the system configuration of the 1st embodimentshown in FIG. 2. Instead, an information separating program 801 storedin the memory unit 105 is used for separating auxiliary information 301from broadcast information 201 received by the broadcast signalreceiving means 101. A method for acquiring auxiliary information 301 inan operation to record video data is also different from that adopted inthe 1st embodiment implementing the broadcast signal receiving apparatus100 shown in FIG. 2.

An operation to record video data carried out in the 2nd embodiment willbe explained. The broadcast signal receiving apparatus 100 receivesbroadcast information 201 and auxiliary information 301 relevantthereto, both of which are transmitted by a broadcasting station. FIG.10 shows the data format of the broadcast information 201 and theauxiliary information 201 received by the 2nd embodiment. Tile auxiliaryinformation 301 is embedded in a program index 203. The processor 102executes the program 801 to extract auxiliary information 301 frominformation received by the broadcast signal receiving means 101 and tostore the information in the storage means 106.

It should be noted that, in some cases, a large amount of auxiliaryinformation 301 can not be transmitted by including the auxiliaryinformation 301 in a program index 203 of broadcast information 201 asshown in FIG. 10 due to broadcasting specifications. In such a case,auxiliary information 301 is transmitted by splitting it into aplurality of packets. Typically, such packets are transmitted in advanceindependently of the associated broadcast information 201 prior to thetime band of the program of the broadcast information 201. The processor102 employed in the broadcast signal receiving apparatus 100 storesauxiliary information 301 included in each packet. As the last packet isreceived, pieces of auxiliary information 301 are arranged in an orderof sequence numbers included in the packets to reconstruct the entireauxiliary information 301. The 2nd embodiment allows the viewer toautomatically acquire auxiliary information 301 by receiving a broadcastsignal without the need to resort to another means such as a recordingmedium used in the 1st embodiment for distributing auxiliary information301.

Next, a 3^(rd) embodiment of the present invention will be explained. Inthe case of the 3^(rd) embodiment, the contents of auxiliary information301 are a CG program, a sentence written in HTML or an operationdescribed as a script. In one typical version of the 3rd embodiment,link information described in HTML is received as auxiliary information301. Video data generated by a source on the Internet is then acquiredby tracing the link and is displayed by executing a browser program. Inanother typical version of the 3^(rd) embodiment, a program forgenerating and displaying CGs (computer graphics) is received asauxiliary information 301. Then, by directly executing this CG program,CG video data can be displayed. FIG. 11 shows the data format ofauxiliary information 301 handled by the 3^(rd) embodiment. As shown inthe figure, an individual auxiliary information piece 304 comprises anauxiliary information index 305 and a program or a script 1001. Theauxiliary information index 305 includes a data type field 1002indicating the data type of the program or the script 1001. Whenexecuting a program or a script 1001, the processor 102 refers to thedata type field 1002 and adjusts the execution of the program or thescript 1001 to the data type specified in the field 1002.

Unlike an operation to play back video and audio data, the executiontime of a program or a script greatly depends on the performance of theprocessor 102 carrying out the processing and the complexity of thenetwork including the data source indicated by a link specified asauxiliary information 301. It is thus difficult to estimate the lengthof the execution time. For this reason, the auxiliary information index305 includes a processing time field 1003. A positive value set in thisprocessing time field 1003 is an effective period of time during which aresult of execution of the program or the script 1001 is to bedisplayed. When the period of time lapses, the operation of thebroadcast signal receiving apparatus is automatically switched to anoperation to play back the video and audio data 202 of the broadcastinformation 201 of the program. On the other hand, a negative value setin this processing time field 1003 indicates that the execution of theprogram or the script 1001 is to be continued up to an instruction toend the execution is issued by the viewer. Thus, in the case of apositive value set in this processing time field 1003, the processor 102keeps monitoring the execution time while the program or the script 1002of auxiliary information 201 is being executed. By virtue of the 3^(rd)embodiment, it becomes possible to provide the viewer with auxiliaryinformation 301 including computer graphics, numbers, texts and figuresin addition to video data by using various kinds of media.

The present invention makes a package media interlockedbroadcasting/watching/listening system feasible. In the package mediainterlocked broadcasting/watching/listening system auxiliary informationrelevant to broadcast information is distributed by using for examplepackage media and, by playing back and displaying the auxiliaryinformation and the broadcast information synchronously, the broadcastinformation can be supplemented with the auxiliary information.

What is claimed is:
 1. A video information displaying method,comprising: receiving a program or a script via a first transmissionpath; receiving video information via a second transmission path;starting to display the video information at a first point in time;executing the program or the script to obtain information via a networkat a second point in time; and resulting in one situation, of aplurality of situations, based on a state of the network, the pluralityof situations including: a first situation in which the informationobtained via the network is displayed at least as a part of a displayimage simultaneously with at least a part of the video information; anda second situation in which execution of the program or the script toobtain the information for display is stopped.
 2. A video informationdisplaying method according to claim 1, wherein stopping the executionof the program or the script is conducted even if an instruction forstopping the execution of the program or the script is not input.
 3. Avideo information displaying method according to claim 2, wherein theinformation obtained via the network includes computer graphics,numbers, texts or figures.
 4. A video information displaying methodaccording to claim 2, wherein the first transmission path includes anetwork path, and the second transmission path includes a broadcastpath.
 5. A video receiving apparatus, comprising: a receiver orreceivers to receive a program or a script via a first transmission pathand to receive video information via a second transmission path; aprocessor to execute the program or the script; and a video datagenerator to generate video data to be displayed, wherein the video datagenerator generates video data including the video information at afirst point in time, wherein the processor executes the program or thescript to obtain information via a network at a second point in time,and wherein operation of the video data generator and the processorresults in one situation, of a plurality of situations, based on a stateof the network, the plurality of situations including: a first situationin which the information obtained via the network is included in thevideo data generated by the video data generator as a part of a displayimage in the video data in order for the information to be displayedsimultaneously with at least a part of the video information; and asecond situation in which the processor stops execution of the programor the script to obtain the information for display.
 6. A videoreceiving apparatus according to claim 5, further comprising: an inputunit to input an instruction from a user, wherein the processor stopsthe execution of the program or the script even if an instruction forstopping execution of the program or the script is not inputted.
 7. Avideo receiving apparatus according to claim 6, wherein the informationobtained via the network includes computer graphics, numbers, texts orfigures.
 8. A video receiving apparatus according to claim 6, whereinthe first transmission path includes a network path, and the secondtransmission path includes a broadcast path.
 9. A display apparatus,comprising: a receiver or receivers to receive a program or a script viaa first transmission path and to receive video information via a secondtransmission path; a processor to execute the program or the script; anda display unit capable of displaying the video information, wherein thedisplay unit displays the video information at a first point in time,wherein the processor executes the program or the script to obtaininformation via a network at a second point in time, wherein operationof the display unit and the processor results in one situation based ona state of the network, among a plurality of situations including: afirst situation in which the display unit displays the informationobtained via the network at least as a part of a display imagesimultaneously with at least a part of the video information; and asecond situation in which the processor stops execution of the programor the script the information for display.
 10. A display apparatusaccording to claim 9, further comprising: an input unit to input aninstruction from a user, wherein the processor stops the execution ofthe program or the script even if an instruction for stopping executionof the program or the script is not inputted.
 11. A display apparatusaccording to claim 10, wherein the information obtained via the networkincludes computer graphics, numbers, texts or figures.
 12. A displayapparatus according to claim 10, wherein the first transmission pathincludes a network path, and the second transmission path includes abroadcast path.